Point reposition depth mapping

ABSTRACT

A method and apparatus for providing optimal correction to depth mapping between captured and displayed stereoscopic content. The solution is derived in a continuous form that can be implemented through CGI scaling techniques compatible with image rendering techniques. Similar correction can be implemented with variable depth-dependent camera separation and disparity re-mapping. The latter is applicable to correcting existing stereoscopic content.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is (1) a continuation-in-part application and claimspriority to U.S. patent application Ser. No. 12/573,852, filed Oct. 5,2009 entitled “Optimal depth mapping,” which is a non-provisionalapplication of U.S. Provisional Patent Application No. 61/102,493, filedOct. 3, 2008 entitled “Optimal Depth Mapping,” and acontinuation-in-part application of U.S. patent application Ser. No.12/541,902, filed Aug. 14, 2009 entitled “Stereoscopic depth mapping”(hereinafter “Stereoscopic depth mapping application”), which is anon-provisional application of U.S. Provisional Patent Application No.61/089,018, filed Oct. 14, 2008 entitled “Linear stereoscopic depthmapping” and (2) a non-provisional application and claims priority toU.S. Provisional Patent Application No. 61/162,280, filed Mar. 21, 2009entitled “Point reposition depth mapping”, all of which arecommonly-assigned and incorporated herein by reference for all purposes.Pursuant to 37 CFR 1.7 (b), this application is hereby filed on Monday,Mar. 22, 2010, which is the next succeeding business day following theone year anniversary of the filing of Prov. Pat. App. No. 61/162,280.

TECHNICAL FIELD

This disclosure generally relates to stereoscopic image processing and,more specifically, to modification of perceived stereoscopic depth bycomputer graphical (CG) modification of point repositioned models.

BACKGROUND

Stereoscopic capture and viewing has been commonplace since CharlesWheatstone invented the Stereoscope in 1833 as discussed in On SomeRemarkable, and Hitherto Unobserved Phenomena of Binocular Vision (Partthe First), Wheatstone, Charles, Phil. Trans. Roy. Soc. Lon. pp. 371-394(1838). Similar to the present day ViewMaster®, this Victorian deviceuses two still camera images which are seen independently by a viewer'seyes. The spatial separation or disparity between similar objects withinthe images gives the sensation of depth.

Capture of stereoscopic content is achieved with two cameras whose fixedspacing is chosen to match a depth budget associated with a playbackdisplay device. This generally results in perceived depth being bothcompressed and distorted producing ‘cardboard cut-outs’ and flat‘wall-paper’ backgrounds.

BRIEF SUMMARY

Methods and apparatuses disclosed in the present application includesubstantially correcting the distorted depth mapping of stereoscopiccapture and display systems.

According to an aspect, a method is provided for generating an optimalstereoscopic image. The method includes an optimizing sequencecomprising determining an original position of a point in a scene havingan original depth, modifying the original position of the point in thescene to a new position having a new depth. According to an aspect, themodifying the original position of the point in the scene to a newposition includes using a depth scaling function, Z′(Z), wherein Zrepresents the original position of the point in the scene. According toanother aspect, the modifying the original position of the point in thescene to a new position further comprises moving the original positionof the point in the scene along a line extending through the originalposition of the point in the scene and a midpoint between two capturecamera positions. According to another aspect, the optimizing sequenceis repeated for every point in the scene.

According to another aspect, another method for generating an optimalstereoscopic image is provided. The method includes determining apreserved perspective of a scene and scaling the preserved perspectiveof the scene.

According to yet another aspect, a method for generating a computergenerated imagery scene using optimal depth mapping is provided. Anexemplary method may include real time rendering of CG images.

Other features and aspects will be apparent with reference to thedetailed description, the drawings, and the claims.

BRIEF DESCRIPTION OF THE DRAWINGS

FIGS. 1A and 1B are schematic diagrams illustrating a top view of ascene and a top view of objects as visualized on a display illustratingnon-linear depth mapping, in accordance with the present disclosure;

FIG. 2 is a schematic diagram illustrating the effect of non-lineardepth mapping on a 3D image, in accordance with the present disclosure;

FIG. 3 is a schematic diagram illustrating stereo capture geometry, inaccordance with the present disclosure;

FIG. 4 is a schematic diagram illustrating camera parameters, inaccordance with the present disclosure;

FIG. 5 is a schematic diagram illustrating the viewing geometry ofdisparity on the screen to perceived depth, in accordance with thepresent disclosure;

FIG. 6 is a schematic diagram illustrating the disparity limits for adisplay screen, in accordance with the present disclosure;

FIG. 7 is a graph illustrating typical non-linear mapping betweenperceived depth and actual depth in a stereoscopic replay displaysystem, in accordance with the present disclosure;

FIG. 8 is a graph illustrating a mapping of direct proportionality and adepth-mapping curve, in accordance with the present disclosure;

FIG. 9 is a graph illustrating a scaled scene mapping for optimal depthreconstruction, in accordance with the present disclosure;

FIG. 10 is a graph illustrating optimal depth mapping relative toconventional depth mapping and linear mapping, in accordance with thepresent disclosure;

FIG. 11 is a graph illustrating optimal mapping solutions for variouscapture/replay parameters, in accordance with the present disclosure;

FIG. 12 is a schematic diagram illustrating an exemplary capture scene,in accordance with the present disclosure;

FIG. 13 is a schematic diagram illustrating an uncorrected anaglyphstereoscopic image, in accordance with the present disclosure;

FIG. 14 is a schematic diagram illustrating an optimally correctedanaglyph image, in accordance with the present disclosure;

FIG. 15 is a graph illustrating a typical non-linear mapping betweenperceived depth and actual depth in a stereoscopic replay displaysystem, in accordance with the present disclosure;

FIG. 16 is a schematic diagram illustrating a scene with depth, inaccordance with the present disclosure;

FIG. 17 is a graph illustrating the non-linear mapping relationship anda desired relationship between actual depth and perceived depth, inaccordance with the present disclosure;

FIGS. 18A and 18B are schematic diagrams illustrating a process and amodule for implementing optimal depth correction, in accordance with thepresent disclosure;

FIG. 19 is a flow diagram for real-time optimal depth correction, inaccordance with the present disclosure;

FIG. 20 is a graph illustrating disparity correction for optimal depthmapping, in accordance with the present disclosure;

FIG. 21A is a flow diagram of an exemplary technique 2100 for imagecreation and FIG. 21B is a schematic diagram illustrating an exemplarymodule 2150 for image creation;

FIG. 22 is a schematic perspective diagram illustrating depth budgetreallocation scaling, in accordance with the present disclosure;

FIG. 23 is a schematic diagram illustrating repositioning options forCGI model points to provide modified disparity, in accordance with thepresent disclosure;

FIG. 24 is a flow chart of a method for depth mapping, in accordancewith the present disclosure; and

FIG. 25 is a graph illustrating the effect of scaling the lateral shiftcorrection Δx associated with optimal mapping, in accordance with thepresent disclosure.

DETAILED DESCRIPTION

When two cameras of fixed separation capture a stereoscopic image pairfrom a real scene, the depth on a playback stereo display is non-linear.Uniformly-spaced objects (such as telegraph poles disappearing into thedistance) appear to get closer together the further away they are. Asused herein, the term “camera” refers to either a physical camera or acapture viewpoint in Computer Generated Imagery (CGI) virtual space. Thepresent disclosure may relate to both a real-world capture environmentand a CGI environment.

FIGS. 1A and 1B are schematic diagrams illustrating this depthdistortion phenomenon. FIG. 1A illustrates the top view of a scene 100with stereoscopic cameras 101 and substantially equally-spaced objects103. FIG. 1B illustrates the top of the same scene 150 as visualized ona display. Viewer 151 faces a display with a display plane 155, andperceives the objects 153 at a non-uniform depth.

FIG. 2 is a schematic diagram illustrating a 3-D scene 200 and theeffect of non-linear depth mapping. The ball 202 is in the foreground ofthe scene 200 and appears too close (and thus, appears distorted). Thedepth between the players 208 and 210, who appear in the middle groundof the scene 200, is relatively good. The crowd 204 in the backgroundappears flat and appears similar to a painted backdrop or wallpaper.

The geometries of the camera capture and display playback systems andthe relation between actual and perceived depth have been analyzed bymany individuals, see, e.g., Lipton, Lenny, Foundations of theStereoscopic Cinema, Van Nostrand Reinhold (1982), which is hereinincorporated by reference for all purposes. Mathematical analysisreveals a general difficulty to provide scaled-depth on finite sizeddisplays as large disparities on physically small screens typicallycause viewer discomfort. Scaled-depth is defined herein as whenperceived depth is directly proportional to actual depth, causingobjects to appear at the appropriate depth for their position and size.Others have proposed correcting for this distorted depth by introducingvariable camera separation for discrete depth regions within the scene,see, e.g., U.S. Pat. No. 7,557,824 to Holliman, which is hereinincorporated by reference for all purposes. This approach is applicablewithin the context of computer graphical (CG) generation of content andis routinely used in Hollywood content generation for scene enhancement,as discussed by Rob Engle in Beowulf 3D: A Case Study, Proc. ofSPIE-IS&T Electronic Imaging, SPIE Vol. 6083, 68030R, which is hereinincorporated by reference for all purposes. To date, these approachestypically use artistic decisions to determine where perceived depthshould be emphasized. In the Stereoscopic depth mapping application, analgorithm is disclosed for determining depth allocation automatically,making it highly suitable for automated environments such as thoseemployed in computer games. The present disclosure also provides amethod for determining depth allocation. Thus, a mathematical analysisof the relationship between the capture and replay display geometries isprovided below.

Mathematical Framework

FIG. 3 shows a top-down view of a symmetrical capture set-up 300 withparallel cameras 302, i.e. cameras with parallel optical axessymmetrical around a line 310 running through the object 308 andperpendicular to the object plane 304. Line 306 runs along the opticalaxis of camera 302 and through a central sensor (or the central portionof the sensor) of camera 302 (and is perpendicular to the object plane304). A symmetrical capture set-up 300 is used to avoid verticaldisparity from keystone distortion that is present with non-paralleloptical axes. The distance Z from the cameras 302 to the plane of theobject 304 is related to d/2 and A/2 by:

$\begin{matrix}{\frac{d}{f} = \frac{A}{Z}} & \left( {{Equation}\mspace{14mu} 1} \right)\end{matrix}$where f is the focal length of camera 302, d/2 is the distance from line306 (or the distance from the central sensor), and A/2 is the distancefrom line 310 to line 306.

FIG. 4 is a schematic diagram of a top-down view 400 of a camera 402.The focal length f of the camera 402 is further related to thehalf-angle capture θ of the camera 402 and its sensor size c by thegeometry of FIG. 4, where:c=2f tan θ  (Equation 2).

FIG. 5 is a schematic diagram of a top-down view of replay geometry 500.A viewer 502 with eye separation E looks at a screen 504 onto which thestereoscopic imagery captured in FIG. 3 is replayed. The distance of thescreen s, the eye separation E, and the screen disparity Ds, determinethe perceived depth σ, through the relation:

$\begin{matrix}{\sigma = {\frac{E \cdot s}{\left( {E - {Ds}} \right)}.}} & \left( {{Equation}\mspace{14mu} 3} \right)\end{matrix}$

The disparity Ds of any one point on the screen from an objectoriginally captured at a distance Z is related to the scaled camerasensor disparity d and a fixed disparity (or offset) D as follows:

$\begin{matrix}{{Ds} = {D - {\frac{W}{c}{d.}}}} & \left( {{Equation}\mspace{14mu} 4} \right)\end{matrix}$

D is the fixed disparity or offset between a left and right image pairsin units of the screen. An image pair may be shifted by a fixeddisparity D after capture. Alternatively, the camera sensors may bemoved relative to the lenses. D is introduced to set the zero parallaxor screen depth position by shifting and cropping of the capturedimages.

Combining Equations 1-4 yields the following expression relatingperceived depth σ with actual depth Z:

$\begin{matrix}{\sigma = {\frac{s \cdot E \cdot Z}{\left( {{E \cdot Z} - {D \cdot Z} + \frac{A \cdot Z}{2\tan\;\theta}} \right)}.}} & \left( {{Equation}\mspace{14mu} 5} \right)\end{matrix}$

This is the general form of the mapping equation relating actual depth Zto perceived depth σ for a given capture-replay system.

Suitable values for A and D are those that fit the mapped depth withinthe limits or disparity budget of the display. All displays have adisparity budget where image separation does not exceed a maximumDs_(max) or a minimum amount Ds_(min).

FIG. 6 is a schematic diagram illustrating a disparity budget for a 3Dimage scene 600 displayed on a display screen 610. The left eye 608 andright eye 606 views for the nearest object 602 and the farthest object604 are shown in the scene 600. The nearest object 602 defines theminimal image separation Ds_(min) between the left eye 608 and the righteye 606 for the scene 600. The farthest object 604 defines the maximumimage separation Ds_(max) between the left eye 608 and the right eye 606for the scene 600. The disparity budget depends on screen size, positionof objects on the screen (e.g., central positioning is more forgivingthan side positioning), and the personal characteristics of the viewer(some people can endure more than others). However, applying maximum andminimum conservative limits to any given display should ensure a goodstereoscopic viewing experience.

Once a disparity budget is determined, a fixed camera separation A andfixed disparity D can be determined mathematically. Using Equations 1-4,the following expressions are derived for the minimum (Ds_(min)) andmaximum (Ds_(max)) on-screen disparities, which correspond to thegreatest separation of near and far objects respectively:

$\begin{matrix}{{Ds}_{\min} = {D - {\frac{W \cdot A}{2Z_{\min}\tan\;\theta}\mspace{14mu}{and}}}} & \left( {{Equation}\mspace{14mu} 6} \right) \\{{Ds}_{\max} = {D - {\frac{W \cdot A}{2Z_{\max}\tan\;\theta}.}}} & \left( {{Equation}\mspace{14mu} 7} \right)\end{matrix}$

Subtracting these two expressions and solving for the camera separationA yields the following expression for a preferred fixed separationcamera set up to accommodate a desired disparity budget:

$\begin{matrix}{A = \frac{{2 \cdot \tan}\;{\theta \cdot Z_{\max} \cdot Z_{\min} \cdot \left( {{Ds}_{\max} - {Ds}_{\min}} \right)}}{W \cdot \left( {Z_{\max} - Z_{\min}} \right)}} & \left( {{Equation}\mspace{14mu} 8} \right)\end{matrix}$which when substituted into Equation 5, yields the following expressionfor the fixed disparity setting:

$\begin{matrix}{D = {\frac{\left( {{Z_{\max} \cdot {Ds}_{\max}} - {Z_{\min} \cdot {Ds}_{\min}}} \right)}{\left( {Z_{\max} - Z_{\min}} \right)}.}} & \left( {{Equation}\mspace{14mu} 9} \right)\end{matrix}$Scaled Mapping

This disclosure provides further information relating to theStereoscopic depth mapping application and provides solutions to thenon-linear depth representation in stereoscopic displays. Thisdisclosure provides an optimal solution in the form of a scalingexpression which may, for example, be used with real-time game renderingtechniques.

As discussed above, the Stereoscopic depth mapping application describesthe non-linear relation between the perceived depth σ and the actualdepth Z of objects captured with a stereo camera pair (camera separationAc, half angle capture θ) when viewed at a distance s from a W widestereoscopic display system. The expression is:

$\begin{matrix}{{\sigma(Z)} = \frac{s \cdot E \cdot Z}{\left( {{E \cdot Z} - {D \cdot Z} + \frac{{Ac} \cdot W}{{2 \cdot \tan}\;\theta}} \right)}} & \left( {{Equation}\mspace{14mu} 10} \right)\end{matrix}$where E is the inter-ocular separation (e.g., 65 mm) and D is the fixeddisparity of the captured images provided either by offsetting thecamera lenses during capture, or image shifting and cropping after thefact. The magnitudes of the camera separation Ac and the fixed disparityD are given by:

$\begin{matrix}{{Ac} = {\frac{2 \cdot \left( {{D\_ max} - {D\_ min}} \right) \cdot {\tan(\theta)} \cdot {Z\_ max} \cdot {Z\_ min}}{W \cdot \left( {{Z\_ max} - {Z\_ min}} \right)}\mspace{14mu}{and}}} & \left( {{Equation}\mspace{14mu} 11} \right) \\{\mspace{20mu}{{D = {{D\_ max} + \frac{W \cdot {Ac}}{2 \cdot {Z\_ max} \cdot {\tan(\theta)}}}}\mspace{20mu}{D = {{D\_ min} + {\frac{W \cdot {Ac}}{2 \cdot {Z\_ min} \cdot {\tan(\theta)}}.}}}}} & \left( {{Equation}\mspace{14mu} 12} \right)\end{matrix}$

In the Stereoscopic depth mapping application, an analytic solution fora variable separation correction resulting in a linear depth mapping wasderived where:σ=α+β·Z  (Equation 13).The Stereoscopic depth mapping application also provides solutions thatwere between Equation 13 and the default mapping with a choice of exactmapping. Justification for this was the competition between the derivedlinear and the desired ‘scaled mapping’ direct proportionalitycondition: σ∝Z (Equation 14).Optimal Mapping

In the present disclosure, it is proposed that the optimum mappingsubstantially ensures a constant factor between the desired depthmapping, or gradient at any position (σ/Z), and the actual mapping

$\left( \frac{\mathbb{d}\sigma}{\mathbb{d}Z} \right).$In other words, the perceived depth at any part in the scene is aconstant factor of the desired, scaled depth. Furthermore, it recognizesthe equivalence of camera separation to scene scaling, providing amapping within the latter framework, thus making it more compatible withCG image rendering. Ultimately it provides a depth-dependent scalingformula.

FIG. 7 is a graph 700 illustrating the typically non-linear depthmapping relationship 702 between σ and Z described by Equation 10. Thecapture and replay parameters used in exemplary case (and in laterexamples) are given in the following table (Table 1):

TABLE 1 Capture and Replay Parameters. Parameter Value Description Z_min2,000 mm Closest object in the captured scene Z_max 50,000 mm Furthestobject in the captured scene D_min −50 mm Maximum screen disparity(Crossed eyes) D_max 20 mm Maximum disparity (Parallel eyes) W 1,000 mmScreen width s 2,000 mm Viewing distance θ 15° Camera half angle capture

Using Equations 11 and 12, the capture camera separation A and the fixeddisparity D are calculated to be ˜78 mm and ˜23 mm respectively.

However, at any depth Z, a preferred gradient of the curve in FIG. 7 is

$\frac{\sigma(Z)}{Z}$in accordance with the “scaled” condition of Equation 14.

FIG. 8 is a graph 800 illustrating the differences between thesepreferred gradients 812, 814, and 816, or localized depth mapping, andthat which is achieved by simple stereoscopic capture and display 818.Angles between the intersecting lines 812, 814, 816 represent an error,which is difficult to reduce to zero. Thus, as discussed in theStereoscopic depth mapping application, direct proportionality may beachieved only in certain circumstances.

In an embodiment of the present disclosure, an optimum condition thatcan be achieved with typical capture and playback parameters isprovided. This optimum condition may be defined as being when the localdepth representation (or mapping gradient) is in direct proportion tothe desired gradient irrespective of scene position. Visually, thismeans the perceived scene is scaled uniformly in depth throughout.Mathematically, this may be described as:

$\begin{matrix}{{\frac{\mathbb{d}}{\mathbb{d}Z}{\sigma(Z)}} = {\alpha \cdot \frac{\sigma(Z)}{Z}}} & \left( {{Equation}\mspace{14mu} 15} \right)\end{matrix}$where σ′(Z) is a modified mapping function, and α is a proportionalityconstant to be determined.

To obtain an optimal mapping, it is possible to vary the spacing A ofthe capture cameras as a function of depth (A(Z) as discussed in theStereoscopic depth mapping application), but one embodiment alsoformulates a solution in terms of depth-dependent scaling (DDS).

DDS is an approach that maps each point in the 3D scene located at (X,Y, Z), with a reference point being the center of the two stereoscopiccamera lenses, onto a new point (X′, Y′, Z′), where:

$\begin{matrix}{\begin{pmatrix}X^{\prime} \\Y^{\prime} \\Z^{\prime}\end{pmatrix} = {\frac{1}{Z} \cdot \begin{pmatrix}{Z^{\prime}(Z)} & 0 & 0 \\0 & {Z^{\prime}(Z)} & 0 \\0 & 0 & {Z^{\prime}(Z)}\end{pmatrix} \cdot {\begin{pmatrix}X \\Y \\Z\end{pmatrix}.}}} & \left( {{Equation}\mspace{14mu} 16} \right)\end{matrix}$

Since the projection of a scene onto a camera sensor remainssubstantially unchanged when scaled in this manner from the camera lensposition, DDS simply affects depth for stereoscopic camera capture.Mathematically, the scaling approach and the camera separation approachare equivalent with a simple relationship:

$\begin{matrix}{\frac{A(Z)}{Ac} = {\frac{Z}{Z^{\prime}(Z)}.}} & \left( {{Equation}\mspace{14mu} 17} \right)\end{matrix}$

To minimize scaling, the scene limits are fixed at Z_max and Z_min andremain substantially the same after scaling. With this desirablerestriction, it is possible to get an optimal analytic mapping functionas follows.

A scaling function Z′(Z) may be used that solves Equation 15 under fixedscene limit conditions. This may be described mathematically as:

$\begin{matrix}{{\frac{\mathbb{d}}{\mathbb{d}Z}{\sigma(Z)}} = {{\frac{\mathbb{d}}{\mathbb{d}Z}{\sigma\left( Z^{\prime} \right)}} = {{\alpha \cdot \frac{\sigma(Z)}{Z}}\mspace{14mu}{where}\text{:}}}} & \left( {{Equation}\mspace{14mu} 18} \right) \\{{Z^{\prime}({Z\_ min})} = {{{Z\_ min}\mspace{14mu}{and}\mspace{14mu}{Z^{\prime}({Z\_ max})}} = {{Z\_ max}.}}} & \left( {{Equation}\mspace{14mu} 19} \right)\end{matrix}$

Equation 10 may then be re-written as:

$\begin{matrix}{{\sigma(Z)} = \frac{A \cdot Z}{Z + B}} & \left( {{Equation}\mspace{14mu} 20} \right) \\{{{with}\mspace{14mu} A} = {{\frac{s \cdot E}{\left( {E - D} \right)}\mspace{14mu}{and}\mspace{14mu} B} = \frac{W \cdot {Ac}}{2 \cdot \left( {E - D} \right) \cdot {\tan(\theta)}}}} & \left( {{Equation}\mspace{14mu} 21} \right)\end{matrix}$Using Equation 20, Equation 18 may be rewritten as:

$\begin{matrix}{{\frac{\mathbb{d}}{\mathbb{d}Z}{\sigma(Z)}} = {{{\left( {\frac{\mathbb{d}}{\mathbb{d}Z^{\prime}}{\sigma\left( Z^{\prime} \right)}} \right) \cdot \frac{\mathbb{d}}{\mathbb{d}Z}}{Z^{\prime}(Z)}} = {\frac{A \cdot \alpha}{Z + B}.}}} & \left( {{Equation}\mspace{14mu} 22} \right)\end{matrix}$And since this is separable both sides can be integrated:

$\begin{matrix}{{\int{\left( {\frac{\mathbb{d}}{\mathbb{d}Z^{\prime}}{\sigma\left( Z^{\prime} \right)}} \right){\mathbb{d}Z^{\prime}}}} = {\int{\frac{A \cdot \alpha}{Z + B}{\mathbb{d}Z}}}} & \left( {{Equation}\mspace{14mu} 23} \right)\end{matrix}$which yields:

$\begin{matrix}{\frac{A \cdot Z^{\prime}}{Z^{\prime} + B} = {{A \cdot \alpha \cdot {\ln\left( {Z + B} \right)}} + \beta}} & \left( {{Equation}\mspace{14mu} 24} \right) \\{{or}\text{:}} & \; \\{{Z^{\prime}(Z)} = {\frac{\left( {{A \cdot \alpha \cdot {\ln\left( {Z + B} \right)}} + \beta} \right) \cdot B}{A - {A \cdot \alpha \cdot {\ln\left( {Z + B} \right)}} - \beta}.}} & \left( {{Equation}\mspace{14mu} 25} \right)\end{matrix}$

Next, using the conditions of Equation 21, α and β can be solved for:

$\begin{matrix}{{\alpha = {\frac{B \cdot \left( {{Z\_ min} - {Z\_ max}} \right)}{{\ln\left\lbrack \frac{\left( {{Z\_ min} + B} \right)}{\left( {{Z\_ max} + B} \right)} \right\rbrack} \cdot \left\lbrack {\left( {{Z\_ min} + B} \right) \cdot \left( {{Z\_ max} + B} \right)} \right\rbrack}\mspace{14mu}{and}}}\mspace{20mu}{\beta = {\frac{A \cdot {Z\_ min}}{{Z\_ min} + B} - {A \cdot \alpha \cdot {{\ln\left( {{Z\_ min} + B} \right)}.}}}}} & \left( {{Equation}\mspace{14mu} 26} \right)\end{matrix}$Equations 25 and 26 then provide an embodiment of the scaling used foroptimal depth representation.

FIG. 9 is a graph 900 representing the scene mapping relationship Z′(Z)902 derived for the parameters of Table 1.

FIG. 10 is a graph 1000 illustrating an optimal depth mapping solution1002 when scene mapping relationship 902 (from FIG. 9) is combined withEquation 20. Optimal depth mapping 1002 (or σ((Z′(Z))) is shown relativeto uncorrected or conventional depth mapping 1004 (or σ(Z)) and linearor “scaled mapping” 1006 (or lin(Z)). The optimal depth mapping solution1002 shows a close relationship to the scaled mapping solution 1006 ofthe Stereoscopic depth mapping application as expected since practicalimplementation suggests that scaled mapping 1006 is closely correlatedwith an optimal depth mapping solution 1002.

FIGS. 11A to 11D are graphs 1110, 1120, 1130, 1140 illustrating tripletcurve sets similar to the graph of FIG. 10. The curve sets of FIGS.11A-11D illustrate mapping solutions for various Capture/Replayscenarios which show the general trend whereby optimal solutions 1112,1122, 1132, 1142 lie between uncorrected solutions 1114, 1124, 1134,1144 and scaled mapping solutions 1116, 1126, 1136, 1146.

FIG. 12 is a schematic diagram illustrating an exemplary stereoscopiccapture scene 1200 including a top scene view 1220 and a camera view1210. A series of balls 1202 in five uniformly spaced depth planes 1204are captured by stereo cameras 1206. In the camera view 1210, the ballsize relates to the distance from the camera 1206.

FIG. 13 is a schematic diagram illustrating an uncorrected stereoscopicimage 1300. In this embodiment, for viewing purposes, stereoscopic image1300 is rendered in anaglyph so that depth can be sensed using classicred cyan glasses.

FIG. 14 is a schematic diagram illustrating an optimal depth mappingstereoscopic image 1400. In this embodiment, for viewing purposes,stereoscopic image 1400 is rendered in anaglyph so that depth can besensed using classic red cyan glasses. By comparison, the images 1300and 1400 illustrate how the optimal depth mapping substantially avoidsthe flattening of the back images relative to those closer to the viewerwithout causing any substantial distortions.

FIG. 15 is another graph 1500 also illustrating the non-linear mappingrelationship 1502 between actual depth Z and perceived depth σ forstereoscopic images. As discussed above, the restraining of scene depthsto within limits based on display sizes (for D≠E) results in anon-linear relation between perceived and actual depth described byEquation 27. Such perceived depth limits are shown by boundary lines formaximum perceived depth σ_max 1504 and minimum perceived depth σ_min1506. Actual depth boundary limits are shown by maximum depth Z_max 1508line and minimum depth Z_min 1510 line. The local perceived depth isdirectly related to the gradient of this curve since a small object atdepth Z of actual thickness t has a perceived thickness τ of:

$\begin{matrix}{\tau = {t \cdot {\left( {\frac{\mathbb{d}}{\mathbb{d}Z}{\sigma(Z)}} \right).}}} & \left( {{Equation}\mspace{14mu} 27} \right)\end{matrix}$

Accordingly, the steeper gradient of line 1502 for close objects impliesa greater depth is experienced relative to those objects further away.As discussed above, for very far objects, the perceived depth is oftenflat, explaining mathematically the phenomenon of painted backdropsoften seen in stereoscopic replay imagery. Of the limited depth budget,most is allocated to the foreground. Depth budget reallocation (DBR)describes altering the perceived/actual depth curve to provide moreuniform local depth throughout the scene.

FIG. 16 is a schematic perspective drawing 1600 illustrating perceiveddepth of two objects and the display heights of those two objectsrespective to one another. A desired perceived depth mapping is one thataccurately preserves perspective, defining a substantially fixedrelation between perceived depth and displayed object heights. Anexample image is shown in drawing 1600 of two similar cars 1602 and 1604captured at different perceived depths (σ₁ and σ₂) from a viewingposition. For these images to appear ‘correct’ (i.e. as if you wereactually viewing two cars 1602, 1604), the relative heights (h₁ and h₂)and perceived depths (σ₁ and σ₂) should be directly proportionalaccording to the following equation:

$\begin{matrix}{\frac{\sigma_{1}}{h_{1}} = {\frac{\sigma_{2}}{h_{2}}.}} & \left( {{Equation}\mspace{14mu} 28} \right)\end{matrix}$

In accordance with the above Equation 28, perceived depth is directlyproportional to actual depth.

FIG. 17 is a graph 1700 illustrating a line 1702 showing the non-linearmapping relationship between actual depth Z and perceived depth σ forstereoscopic images and a line 1704 showing a desired relationshipbetween perceived and actual depth. This desired direct proportionalrelationship is represented by a straight line 1704 passing through theorigin on the perceived vs. actual depth graph 1700. Generally, smallerdisplays such as LCD monitors have smaller disparity limits than largescreens or movie screens, therefore it is difficult to provide such adesired mapping between perceived depth and actual depth for someperspective views on such smaller displays. While it is possible toattempt preservation of this directly proportional relationship, displayof such on smaller displays may lead to viewer fatigue and headaches.

As discussed above, to address this problem and to make the stereoscopicview more natural, the present disclosure provides for an altering ofthe relation between the perceived and actual depth through a depthscaling function Z′(Z). Below, another embodiment of a depth scalingfunction Z′(Z) is discussed.

Depth Scaling Function

The depth scaling function Z′(Z) can be understood as a modified depthfor each point or vertex, initially positioned at Z, making up all theobjects in a scene. To retain an object's captured size, each point maybe reassigned a depth value Z′ and may also be scaled by the ratio ofnew to old depth toward the camera. Mathematically, the depth values forpoints at (x, y, z) co-ordinates in a three-dimensional space may betransformed to reassigned depth values at points (x′, y′, z′) inaccordance with the following equation:

$\begin{matrix}{\begin{pmatrix}x^{\prime} \\y^{\prime} \\Z^{\prime}\end{pmatrix} = {\frac{Z^{\prime}(Z)}{Z} \cdot \begin{pmatrix}x \\y \\z\end{pmatrix}}} & \left( {{Equation}\mspace{14mu} 29} \right)\end{matrix}$where the system coordinates' origin is the point midway between thecamera lenses.

FIGS. 18A and 18B are schematic diagrams illustrating an embodiment of aprocess 1800 and a module 1850 for implementing an optimal depth mappingalgorithm. In the process 1800, the original position of a point

$\quad\begin{bmatrix}x \\y \\z\end{bmatrix}$(discussed above as an initial position Z) is determined in step 1802.The original position of the point is modified by some scaling solutionZ′(Z) in step 1804. The original position may be normalized or scaled by1/Z in step 1808 and a new position

$\quad\begin{bmatrix}x^{\prime} \\y^{\prime} \\z^{\prime}\end{bmatrix}$is determined at step

In an embodiment, the process 1800 is implemented in a scaling module1850 operable to receive and analyze scene and display information 1852and provide modified scene information 1856. The module 1850 may be anysuitable logical device known in the art, and may be embodied inhardware, software, or a combination. The module 1850 may also be storedon a computer readable medium, as instructions for a machine.Determining the scaling solution Z′(Z) to render the new position

$\quad\begin{bmatrix}x^{\prime} \\y^{\prime} \\z^{\prime}\end{bmatrix}$an optimal depth mapping solution is discussed below.Local Depth

The local perceived thickness τ of a small object of thickness h isgiven by Equation 30:

$\begin{matrix}{\tau = {h \cdot {\left( {\frac{\mathbb{d}}{\mathbb{d}Z}{\sigma(Z)}} \right).}}} & \left( {{Equation}\mspace{14mu} 30} \right)\end{matrix}$

From Equation 30, a local perceived depth Δ may be defined as being theratio of the real to perceived thickness, according to Equation 31below:

$\begin{matrix}{\Delta = {\frac{\mathbb{d}}{\mathbb{d}Z}{{\sigma(Z)}.}}} & \left( {{Equation}\mspace{14mu} 31} \right)\end{matrix}$

A captured scene, modified through a scaling operation, has a modifiedlocal depth, Δ′, where:

$\begin{matrix}{{\Delta^{\prime}(Z)} = {\frac{\mathbb{d}}{\mathbb{d}Z}{{\sigma\left( {Z^{\prime}(Z)} \right)}.}}} & \left( {{Equation}\mspace{14mu} 32} \right)\end{matrix}$Desired Depth

One-to-one replay would mean the local depth is equal to 1 throughoutthe scene, and, thus, perceived depth limits (determined by the screendisparity limits) will lie on a line 1704 in FIG. 17 where σ=Z. Ingeneral, this becomes difficult, as it creates extreme disparity for theviewer. To address the extreme disparity for the viewer, a desired localdepth may be used as a fixed proportion of the perspective preserving(“scaled mapping” as discussed in the Stereoscopic depth mappingapplication), direct proportional gradient Γ according to Equation 33:

$\begin{matrix}{\Gamma = \frac{\sigma(Z)}{Z}} & \left( {{Equation}\mspace{14mu} 33} \right)\end{matrix}$or in the case where the scene is scaled by Z:

$\begin{matrix}{\Gamma = {\frac{\sigma\left( {Z^{\prime}(Z)} \right)}{Z}.}} & \left( {{Equation}\mspace{14mu} 34} \right)\end{matrix}$

Mathematically, a scaling solution, Z′(Z), is preferred, where for anyposition Z within the scene where the following relation holds:

$\begin{matrix}{{\frac{\mathbb{d}}{\mathbb{d}Z}{\sigma\left( {Z^{\prime}(Z)} \right)}} = {\gamma \cdot \frac{\sigma\left( {Z^{\prime}(Z)} \right)}{Z}}} & \left( {{Equation}\mspace{14mu} 35} \right)\end{matrix}$with γ a constant independent of Z. This can be solved analytically.

First Embodiment of a Scaling Solution Z′(Z)

Equation 35 can be expressed as:

$\left( \begin{matrix}{{{\left. {\frac{\mathbb{d}}{\mathbb{d}Z}{\sigma\left( Z^{\prime} \right)}} \right) \cdot \frac{\mathbb{d}}{\mathbb{d}Z}}Z^{\prime}} = {\gamma \cdot {\frac{\sigma\left( Z^{\prime} \right)}{Z}.}}} & \left( {{Equation}\mspace{14mu} 36} \right)\end{matrix} \right.$

From Equation 36, Equation 37 may be derived:

$\begin{matrix}{{{\sigma\left( Z^{\prime} \right)} = \frac{\alpha}{1 + \frac{\beta}{Z^{\prime}}}}{{{where}\text{:}\mspace{14mu}\alpha} = {{\frac{E \cdot s}{\left( {E - D} \right)}\mspace{14mu}{and}\mspace{14mu}\beta} = {\frac{W \cdot A \cdot f}{\left( {E - D} \right) \cdot c}.}}}} & \left( {{Equation}\mspace{14mu} 37} \right)\end{matrix}$

Substituting this expression into Equation 36 gives:

$\begin{matrix}{\frac{\beta \cdot {\mathbb{d}Z^{\prime}}}{Z^{\prime} \cdot \left( {Z^{\prime} + \beta} \right)} = \frac{\gamma \cdot {\mathbb{d}Z}}{Z}} & \left( {{Equation}\mspace{14mu} 38} \right)\end{matrix}$which, when integrated, gives:ln(Z′)−ln(Z′+β)=γ·ln(Z)+∞  (Equation 39)where γ is a constant of integration.

Equation 39 can be solved to yield an embodiment of a solution for ascaling solution:

$\begin{matrix}{{Z^{\prime}(Z)} = \frac{\beta}{\left( {{K \cdot Z^{- \gamma}} - 1} \right)}} & \left( {{Equation}\mspace{14mu} 40} \right)\end{matrix}$where K is a constant related to γ.

To complete the solution, the constants γ and K are determined byapplying the disparity limit conditions:

$\begin{matrix}{Z_{\max} = {{\frac{\beta}{\left( {{K \cdot Z_{\max}^{- \gamma}} - 1} \right)}\mspace{14mu}{and}\mspace{14mu} Z_{\min}} = {\frac{\beta}{\left( {{K \cdot Z_{\min}^{- \gamma}} - 1} \right)}.}}} & \left( {{Equation}\mspace{14mu} 41} \right)\end{matrix}$

Solving these for γ and K and back substituting for β (Equation 37) andthen for A and B (Equations 8 and 9), the following scaling solutionthat delivers the desired optimal depth mapping profile is derived:

$\begin{matrix}{\mspace{79mu}{{{Z^{\prime}(Z)} = \frac{\beta}{\left( {{K \cdot Z^{- \gamma}} - 1} \right)}}\mspace{79mu}{{where}\text{:}}{\beta = \frac{\left( {D_{\max} - D_{\min}} \right) \cdot \left( {Z_{\min} \cdot Z_{\max}} \right)}{\left\lbrack {{E \cdot \left( {Z_{\max} - Z_{\min}} \right)} - {D_{\max\;} \cdot Z_{\max}} + {D_{\min} \cdot Z_{\min}}} \right\rbrack}}\mspace{79mu}{\gamma = {{\frac{\ln\left\lbrack \frac{\left( {E - D_{\max}} \right)}{E - D_{\min}} \right\rbrack}{\ln\left( \frac{Z_{\min}}{Z_{\max}} \right)}\mspace{14mu}{and}\mspace{14mu} K} = {\frac{\left( {Z_{\min} + \beta} \right)}{Z_{{mi}n}^{1 - \gamma}}.}}}}} & \left( {{Equation}\mspace{14mu} 42} \right)\end{matrix}$

Second Embodiment of a Scaling Solution Z′(Z)

In another embodiment, an alternative scaling solution or pointreposition depth mapping function Z′(Z) may be used. This alternativeembodiment point reposition depth mapping function is defined asfollows:

${{Z^{\prime}(Z)} = \frac{\left( {{\alpha \cdot {\ln\left( {Z + \beta} \right)}} + B} \right) \cdot \beta}{1 - {\alpha \cdot {\ln\left( {Z + \beta} \right)}} - B}},{{{where}\mspace{14mu}\beta} = \frac{\left( {D_{\max} - D_{\min}} \right) \cdot \left( {Z_{\min} \cdot Z_{\max}} \right)}{\left\lbrack {{E \cdot \left( {Z_{\max} - Z_{\min}} \right)} - {D_{\max\;} \cdot Z_{\max}} + {D_{\min} \cdot Z_{\min}}} \right\rbrack}},\text{}{{{where}\mspace{14mu}\alpha} = \frac{\beta \cdot \left( {Z_{{mi}n} - Z_{\max}} \right)}{{\ln\left\lbrack \frac{\left( {Z_{\min} + \beta} \right)}{\left. {Z_{\max} + \beta} \right)} \right\rbrack} \cdot \left\lbrack {\left( {Z_{\min} + \beta} \right) \cdot \left( {Z_{\max} + \beta} \right)} \right\rbrack}},{{{where}\mspace{14mu} B} = {\frac{Z_{\min}}{Z_{\min} + \beta} - {\alpha \cdot {\ln\left( {Z_{\min} + \beta} \right)}}}},$where Z is an initial model point location from a left-eye or aright-eye observation point, where

${Z = \begin{pmatrix}x \\y \\z\end{pmatrix}},$and where E is eye separation. (Equation 43).

This mapping may be a desirable choice in cases where automatedcorrection of depth is desired, such as in computer gaming scenarios orfor automated CGI stereoscopic rendering.

In yet another embodiment, the point reposition depth mapping functionmay be implemented using a look up table, where predetermined values forZ′(Z) are located in the table based upon input parameters. Such a lookup table may be populated with predetermined values calculated using theforegoing examples of point reposition depth mapping functions.

Implementation

In an embodiment, use of optimal mapping within a gaming environment(for example) wherein stereoscopic images are rendered on the fly as afunction of player position and view direction is provided. Inconventional 2D games, it is routine to render a single image based onthe viewer position on a frame by frame basis. Extension to stereo is asimple matter of rendering two views for different camera positions.Fortunately, current commercially available stereo TVs display halfresolution images for left and right eye views making stereo renderingdemands no greater than that of conventional 2D HD. These demandshowever are still such that rasterization techniques are used ratherthan ray tracing.

During rasterization, vertices of polygons that make up the 3D worldmodel are first transformed using matrices that scale, translate, rotateetc. before being projected onto the 2D view plane and shaded inaccordance with their depth (i.e. their Zbuffer values). In anembodiment, a transformation of model polygons in accordance with thenon-linear scaling transformation (Equation 16) during real-timerendering of a stereoscopic game, thus optimizing the 3D experience, isproposed. To achieve this, it is preferable to apply the capture andreplay parameters. Fortunately, many are fixed in any single gamesession. Those that commonly vary include the scene depth limits Z_(max)and Z_(min).

One technique for obtaining these parameters is to use the depthinformation from a previous rendered Zbuffer; whereas another is tocalculate them directly from the polygon vertices data beforetransformation. Object depth information might also be used to alter thedisparity limits dependent on its position within the view. Near objectsclose to or even intersecting the vertical edges of the display might beused to reduce the disparity limits, hence, avoiding undesirable depthcue conflicts. Large changes in parameters should be avoided in the gameauthoring, but if sudden modest jumps occur in the capture parametersthen alteration of the mapping transformation should be damped avoidinglarge jumps in depth.

FIG. 19 is a flow diagram 1900 showing an exemplary process forreal-time optimal depth correction. In step 1901, a frame's position anddirection are determined (this includes determining the coordinates of aparticular vertex of a polygon of the frame). Next, in step 1902, Z_maxand Z_min for that position and direction are determined. From that, Aand D can be calculated using equations 11 and 12 in step 1904. Thescaling function or scaling solution Z′(Z) can then be determined usingequations 25 and 26 in step 1906. Using equation 16, the vertices ofpolygons are transformed in step 1908 using the original informationfrom step 1901 and the scaling function Z′(Z) determined in 1906. Next,the left and right images are rendered in steps 1910 and 1912 and thenbuffered in steps 1914 and 1916 before being displayed in step 1920.Then the process may start over again in step 1901, where the nextframe's position and direction (and the coordinates of a polygon) aredetermined. As discussed above, in an embodiment, the depth informationfrom a previous rendered Zbuffer may be used for determining theposition and direction in step 1901. In another embodiment, depthinformation may be calculated directly from the polygon vertices databefore transformation in determining the position and direction in step1901.

Another embodiment may use the optimal mapping scaling method onnon-real time applications and one that would employ alternativerendering algorithms such as ray tracing. Once again, the 3D model wouldundergo non-linear scaling in accordance with the formulation, andstereoscopic images would be rendered accordingly.

Other embodiments could remap depth in pre-captured images according todisparity. Assuming the capture replay parameters are known, thedisparity Ds between left- and right-eye images relates to perceiveddepth according to:

$\begin{matrix}{\sigma = \frac{s \cdot E}{E - {D\; s}}} & \left( {{Equation}\mspace{14mu} 44} \right)\end{matrix}$Manipulation of this expression and replacing the uncorrected perceiveddepth (σ) with the desired optimal one (σ′) yields the followingexpression for corrected disparity Ds′:

$\begin{matrix}{{Ds}^{\prime} = {E - {E \cdot {\frac{s}{\sigma\left( {Z^{\prime}(Z)} \right)}.}}}} & \left( {{Equation}\mspace{14mu} 45} \right)\end{matrix}$Back substitution gives the following relation between old and newdisparity.

$\begin{matrix}{{Ds}^{\prime} = {E - {E \cdot {\frac{s}{\sigma\left\lbrack {Z^{\prime}\left\lbrack {{- B} \cdot E \cdot \frac{s}{\left( {{E \cdot s} + {A \cdot {Ds}} - {A \cdot E}} \right)}} \right\rbrack} \right\rbrack}.}}}} & \left( {{Equation}\mspace{14mu} 46} \right)\end{matrix}$

This expression allows the disparity Ds of any pixel withinconventionally captured stereoscopic images to be given a new value Ds′.In practice, this means shifting pixel values and smearing orin-painting where gaps are left.

FIG. 20 is a graph 2000 illustrating pixel shift as a function ofmeasured disparity Ds−Ds′ (Ds) 2002 for the capture/replay parameters ofTable 1. Implementation of this embodiment may use disparity estimationusing established or other techniques, shifting pixels according toEquation 44, and then filling in voids using either established orfuture in-painting techniques.

CG Rendering

In an embodiment, the implementation of this reallocation approach iswell-suited to a CG environment where 3D models exist together in a 3Dworld and from which a camera is able to create 2D images of a givenscene.

Image creation is termed rendering and, in its raw form, is amathematical manipulation of the graphical information. As discussedabove, 3D models generally have points or vertices between whichtextures are painted. Each vertex has a coordinate (X, Y, Z) within aCartesian world system described by basis vectors i, j, and k. Eachvertex also typically has a color or a texture associated with it toallow the polygons (defined by the lines joining the vertices) to becorrectly shaded.

A camera is situated within the world at a position (C_(x), C_(y),C_(z)) with frustum orientation vectors I, J, K, where I and J areoriented along the horizontal and vertical aspects of the final image,and K is the direction at which the camera is pointed. The aspect of thecamera is a, where a≈0.56 for standard HD format, and finally it has azoom setting ζ defined here as the ratio c/2f.

In general, a vertex may be shifted along a line extending from itselfto the position of a capturing camera without affecting its 2D imagelocation. To modify disparity for depth correction, an additional shiftmay be used with a component along a direction parallel to the offset ofthe cameras. In this manner, numerous manipulations may yield the samecorrected depth capture, which includes z-scaling.

General depth modifying point repositioning can be defined by:

$\begin{matrix}{\begin{pmatrix}x^{\prime} \\y^{\prime} \\z^{\prime}\end{pmatrix} = {{k \cdot \begin{pmatrix}x \\y \\z\end{pmatrix}} + {\begin{pmatrix}{\Delta\; x} \\0 \\0\end{pmatrix}.}}} & \left( {{Equation}\mspace{14mu} 47} \right)\end{matrix}$where the coordinates have been chosen (quite generally) with the camera(left or right) located at (0; 0; 0), having an optic axis along (0; 0;1), and separated from its partner along an axis parallel to (1; 0; 0).Δx directly affects the disparity of the repositioned point, the extentof which is determined by the value of k.

The varied repositioning options covered by Equation 47 are shownschematically in FIG. 23. The position on the camera sensor of thepoint's modified image may be chosen to adjust the point's disparity forcorrected perceived depth. As shown, each of the repositioned pointsyields the same image on the camera. An original point position 2302 isshown. A possible point reposition that yields the same modified imageare shown at preferred lateral shift position 2304. Another possiblepoint reposition is the strictly smallest shift to point 2306.Alternatively, other exemplary point repositions may be toward points2308, 2310, or any other point located on the line 2320 that providesthe modified image.

Given the infinite possibilities of this correction method, minimizingthe shift toward the camera (i.e., to point 2304) may be chosen as itbest avoids the undesirable issues with lighting and texture mapping.For this reason, in an embodiment, no shift toward the camera with k=0in Equation 47 is chosen. While this is not the minimum shiftpossible—which may be achieved by moving normal to the principal imagingray (i.e., to point 2306)—it does offer effectively the same small shiftwhile allowing more convenient mathematical translation of mappingfunctions.

Another advantage of the subtle lateral shift approach over the moreextreme z-scaling approach is a significant reduction in depth reversalcaused by sparse model points or vertices. Repositioning sparse verticesmay cause points forming the connecting lines to acquire positionsdifferent from those should they have been manipulated individually. Onoccasions, the incorrect positions may lead to depth reversal where aline point can end up in front of other repositioned vertices reversingtheir original depth order. The very small lateral shiftingsubstantially eliminates these problems for almost any scene barringthose using very large polygons. In these cases, lines may be broken byintroducing intermediate points.

In various embodiments, depth correction techniques may be embodiedusing one or more CG modeling packages, e.g., commercially availableproducts Maya or 3D Studio Max, both supplied by Autodesk. Thesepackages may allow additions in the form of plug-ins or scripts thatmanipulate the capture of images. In some embodiments, moresophisticated rendering algorithms such as mental Ray®, and physicsengines such as Havok™ may be incorporated. The additions may offer theoption of rendering a scene, or a sequence of scenes with a pair ofstereo cameras. The capture parameters (camera separation and skews)could either be defined manually or automatically using Equations 11 and48, where skew S may be defined as:

$\begin{matrix}{S:={\frac{2 \cdot {\tan(\theta)} \cdot \left( {{{D\_ max} \cdot {Z\_ max}} - {{D\_ min} \cdot {Z\_ min}}} \right)}{W \cdot \left( {{Z\_ max} - {Z\_ min}} \right)}.}} & \left( {{Equation}\mspace{14mu} 48} \right)\end{matrix}$

For automatic determination, disparity limits D_max and D_min may beused, as they are viewer and display dependent. The scene depth may alsobe manually input, or possibly automatically input by interrogating thedepth of those model points that are visible. In an embodiment, scanningthe z-buffer for the maximum and minimum values is implemented.Additionally, disparity limits may be predetermined based upon the sizeof the viewer's screen and/or screen type.

FIG. 24 is a flow diagram of an exemplary technique 2400 for imagecreation that uses the subtle lateral shift approach described above.Here, the CG model, texture, camera positions, and lighting parametersare input at step 2402. It is then determined whether this is astereoscopic capture at step 2404. If it is not a stereoscopic capture,then the image is rendered as a 2D image, at step 2406. Otherwise, ifstep 2404 determines that it is a stereoscopic capture, then cameraseparation A and skew S parameters are determined at step 2410, alsotaking into consideration disparity limits D_(min), D_(max), Z_(max),and Z_(min), as automatically determined or manually input from step2408. Model points are then shifted according to a desired depth mappingfor the right camera, at step 2412, Similarly, model points are shiftedaccording to a desired depth mapping for the left camera, at step 2414.The 2D right eye image is rendered at step 2416, and the 2D left eyeimage is rendered at step 2418.

FIG. 21A is a flow diagram of an exemplary technique 2100 for imagecreation and FIG. 21B is a schematic diagram illustrating an exemplarymodule 2150 for image creation. Exemplary image creation technique 2100may be performed in a three step process. For instance, in step 2102,the polygon vertices may be transformed into the camera's coordinatesystem—the so-called view transformation. In step 2104, the samevertices may be projected onto an image plane with normalized depthvalues. View transformation 2102 and projection transformation 2104 arediscussed in greater detail below. The pixel values may be calculatedusing the process of rasterization in step 2106. Depth reallocation maybe incorporated during either one of the first two of these three steps.

In an embodiment, the image creation technique 2100 is implemented in animage creating module 2150 operable to receive and analyze image dataincluding input vertices 2152 and provide modified image data includingoutput vertices 2156. The module 2150 may be any suitable logical deviceknown in the art, and may be embodied in hardware, software, or acombination. The module 2150 may also be stored on a computer readablemedium, as instructions for a machine; and may be a plug-in for a 3Dsoftware package.

View Transformation

To define a vertex position in a coordinate system whose origin is atthe camera and whose orientation is defined by the capture frustum,world coordinate values may be translated and rotated. Whiletranslations can be described by 3×3 matrix manipulationstransformations may utilize a 4×4 representation. A general vertexposition may be given by:

$\begin{matrix}{\begin{pmatrix}X \\Y \\Z \\1\end{pmatrix}.} & \left( {{Equation}\mspace{14mu} 49} \right)\end{matrix}$

The unit value of the fourth component can be regarded as a dummyparameter whose value may be chosen to be one.

Translating a general vertex position to one relative to the cameraposition is then a matter of performing the following matrixmanipulation:

$\begin{matrix}{\begin{pmatrix}1 & 0 & 0 & {- {Cx}} \\0 & 1 & 0 & {- {Cy}} \\0 & 0 & 1 & {- {Cx}} \\0 & 0 & 0 & 1\end{pmatrix} \cdot {\begin{pmatrix}X \\Y \\Z \\1\end{pmatrix}.}} & \left( {{Equation}\mspace{14mu} 50} \right)\end{matrix}$

Rotating the resulting vertex representation provides its position(vx,vy,vz) in desired camera view coordinates:

$\begin{matrix}\begin{matrix}{\begin{pmatrix}{vx} \\{vy} \\{vz} \\1\end{pmatrix} = {\begin{pmatrix}{I \cdot i} & {I \cdot j} & {I \cdot k} & 0 \\{J \cdot i} & {J \cdot j} & {J \cdot k} & 0 \\{K \cdot i} & {K \cdot j} & {K \cdot k} & 0 \\0 & 0 & 0 & 1\end{pmatrix} \cdot}} \\{\begin{pmatrix}1 & 0 & 0 & {- {Cx}} \\0 & 1 & 0 & {- {Cy}} \\0 & 0 & 1 & {- {Cz}} \\0 & 0 & 0 & 1\end{pmatrix} \cdot \begin{pmatrix}X \\Y \\Z \\1\end{pmatrix}} \\{= {\begin{pmatrix}{I \cdot i} & {I \cdot j} & {I \cdot k} & {{- I} \cdot C} \\{J \cdot i} & {J \cdot j} & {J \cdot k} & {{- J} \cdot C} \\{K \cdot i} & {K \cdot j} & {K \cdot k} & {{- K} \cdot C} \\0 & 0 & 0 & 1\end{pmatrix} \cdot {\begin{pmatrix}X \\Y \\Z \\1\end{pmatrix}.}}}\end{matrix} & \left( {{Equation}\mspace{14mu} 51} \right)\end{matrix}$Projection Transformation

The generalized perspective transformation can be written:

$\begin{matrix}{\begin{pmatrix}{xx} \\{yy} \\{zz} \\{vz}\end{pmatrix} = {\begin{pmatrix}\frac{2 \cdot {Zn}}{Vw} & 0 & 0 & 0 \\0 & \frac{2 \cdot {Zn}}{Vh} & 0 & 0 \\0 & 0 & \frac{Zf}{{Zf} - {Zn}} & \frac{{Zf} \cdot {Zn}}{{Zf} - {Zn}} \\0 & 0 & 1 & 0\end{pmatrix} \cdot \begin{pmatrix}{vx} \\{vy} \\{vz} \\1\end{pmatrix}}} & \left( {{Equation}\mspace{14mu} 52} \right)\end{matrix}$where:

Zn is the distance from camera to the plane where Zbuffer value=0(typically near clipping plane),

Zf is the distance from camera to the plane where Zbuffer value=1(typically far clipping plane),

Vh is the height of the viewport positioned at Zn, and

Vw is the width of the viewport positioned at Zn.

The normalized (x′, y′) position of the vertex in the image plane isthen:

$\begin{matrix}{\left( {x^{\prime},y^{\prime}} \right) = \left( {\frac{xx}{vz},\frac{yy}{vz}} \right)} & \left( {{Equation}\mspace{14mu} 53} \right)\end{matrix}$with its Zbuffer value given by:

$\begin{matrix}{{Zbuffer} = {\frac{\left( {{vz} - {Zn}} \right) \cdot {Zf}}{\left( {{Zf} - {Zn}} \right) \cdot {vz}}.}} & \left( {{Equation}\mspace{14mu} 54} \right)\end{matrix}$Depth Budget Reallocation Using World Scaling

In an embodiment, depth budget reallocation is implemented prior to thefirst view transformation and is carried out one time before a left anda right eye capture. The method scales the vertex positions (x,y,z)toward the midpoint (mx, my, mz) between lenses of the camera rig and isreferred to herein as world scaling.

FIG. 22 is a schematic perspective diagram 2200 illustrating worldscaling. A frustum far plane 2214, a frustum close plane 2210, and ascreen or image plane 2212 are shown in the diagram 2200. The originalobject 2202 is replaced by a scaled replacement object 2204. Theoriginal object 2202 is at an original depth Z and has a screendisparity of Ds. The original object 2202 may be scaled toward thecamera midpoint 2206 along a line 2220. The camera midpoint 2206 is themid-point between the camera positions 2208 and the line 2220 extendsfrom the original object 2202 through the camera midpoint 2206. Thescaled replacement object 2204 is at a new depth Z′ and has a screendisparity of Ds′.

Mathematically, this can be written with the same 4D representation as acoordinate origin shift, a scaling and inverse shift, or:

$\begin{matrix}{\left\lbrack {\begin{pmatrix}{I \cdot i} & {I \cdot j} & {I \cdot k} & {{- I} \cdot C} \\{J \cdot i} & {J \cdot j} & {J \cdot k} & {{- J} \cdot C} \\{K \cdot i} & {K \cdot j} & {K \cdot k} & {{- K} \cdot C} \\0 & 0 & 0 & 1\end{pmatrix}^{- 1} \cdot \begin{pmatrix}1 & 0 & 0 & 0 \\0 & 1 & 0 & 0 \\0 & 0 & 1 & 0 \\0 & 0 & 0 & \frac{Z}{Z^{\prime}(Z)}\end{pmatrix} \cdot \begin{pmatrix}{I \cdot i} & {I \cdot j} & {I \cdot k} & {{- I} \cdot C} \\{J \cdot i} & {J \cdot j} & {J \cdot k} & {{- J} \cdot C} \\{K \cdot i} & {K \cdot j} & {K \cdot k} & {{- K} \cdot C} \\0 & 0 & 0 & 1\end{pmatrix}} \right\rbrack \cdot {\begin{pmatrix}X \\Y \\Z \\1\end{pmatrix}.}} & \left( {{Equation}\mspace{14mu} 55} \right)\end{matrix}$

The resulting vectors are scaled to make their fourth component equal toone.

Application to Computer Game and CG Environments

In various embodiments, depth-mapping within a real-time computer gameor CG-created graphical environment (e.g., a flight simulator, virtualreality environment, etc.) may be employed. The basic implementation issimilar to those already disclosed above, and it is compatible with arasterization technique of image formation. Automation is preferred forinteractive games making z-Buffer determination of scene depth andoptimal mapping preferred. It is assumed that real-time stereoscopicrendering preferably preserves operating frame rate and transmissionbandwidth. A suitable solution is provided for by RealID's output dataformat (e.g., as taught in commonly-owned Pat. App. Ser. No. 61/168,925,herein incorporated by reference), where half resolution images arerendered on a diagonal grid to produce side-by-side single stereoscopicframes for transmission down standard DVI cabling.

The dynamic response used by games differs from the CG 2D capture.Automated correction may be used to adjust the depth mapping on a frameby frame basis. Simple implementations using correction parametersdetermined from single frames may cause problems. For example, closeobjects entering the scene may cause rapid changes in perceived deptheven prior to the viewer noticing objects entering from the side. Toavoid this and related phenomena, the adjustment may incorporate eitherdamped adjustment by possibly taking an average of scene parameters overmany frames, or adjusting slowly to anticipated events. The latter couldbe based on the proximity of depth adjusting objects to the capturefrustum.

Edge conflict may also be an issue with games which, though a problemwith non real-time CG capture, may be dealt with automatically within agame scenario, especially if desirable negative disparity effects areallowed. Edge conflict is when a negative disparity object (i.e., onethat appears in front of a stereoscopic display) is apertured by thevertical screen edge. The object may appear to be visually blocked bysomething behind it. This may cause confusion and discomfort to theviewer. One solution, as disclosed by Robert Neuman in Bolt 3D: A CaseStudy, involves floating windows where a false edge is displayed withthe object which also has negative disparity avoiding the conflict. In agame, introducing floating windows dynamically may become intrusive. Abetter option is to substantially prevent objects from having negativedisparity close to the edge by either (1) not allowing negativedisparity; (2) only allowing negative disparity when there is noconflict; or (3) altering the disparity closer to the edges. The last isnot favored as it may cause noticeable depth distortion in the form offish bowl distortion. On the other hand, the first detracts from the“wow” factor that gamers desire. The second option is therefore favored,and may be incorporated into the overall depth mapping algorithm.

A further consideration for stereoscopic computer gaming is viewer headmovement and tilt. Unlike movies in the cinema, a computer game invitesthe participant to move physically as part of the experience. Nintendohas recently capitalized on this with its highly commercially successfulWii game console. Moving viewing position causes the displayed world torotate unnaturally about a vertical axis situated in the plane of thescreen. This is a consequence of the viewing geometry and not wellappreciated by most whose primary experience is in viewing 3D films at alarge distance from a cinema screen. Here, the rotation is too slight tobe unnoticed. To avoid rotation on the smaller gaming displays, imagesmay be altered in accordance with the viewer's position. While headtracking can be done with direct camera imaging of the player, it issimplified in cases where the viewer wears recognizable eyewear. Manyapproaches can be considered, although a passive infrared illuminationof retro-reflector adorned eyewear is possibly preferred (e.g., astaught in commonly-owned patent application Ser. No. 12/606,828, hereinincorporated by reference). Once detected, the head position requiresthe imagery to be modified to avoid unnatural 3D rotation. Themodification can be determined empirically with trial and error methods,although it can also be calculated accurately based on the knownperceived depth. This latter is preferred as it avoids sensory conflictsand subsequent fatigue.

Head tilt can be determined using the same head tracking hardware. Thisis convenient, as stereoscopic imagery may not be viewed correctly whenviewed with a tilted head. Images typically have horizontal disparitywhich is only correct for viewers whose eyes are horizontal. At anangle, the display appears to have disparity normal to the axis of theeyes, which is known to cause fatigue. Offsetting the imagery at thesame angle as the viewer's head avoids this issue and leads to a moreimmersive experience. The point repositioning depth correction may bemodified in accordance with the rotated disparity axis.

Games might also desire unnatural depth adjustment to exaggerateeffects. Modification to the optimal mapping could accomplish this(together with other mapping functions Z′(Z)) through simple scaling ofthe lateral scaling function Δx(Z). Mathematically, a scaling value ucould be introduced where a new lateral displacement Δx′(Z)=υ.Δx(Z)could be implemented which alters the correction as desired.

FIG. 25 is a graph 2500 illustrating the effect of scaling the lateralshift correction Δx associated with optimal mapping, showing the effectof υ on optimal mapping. As shown, the gradient of the curves movedeeper as the value of υ increases. For the viewer, the more distantareas of the scene get stretched which could be desired. The moreseparated in depth a region becomes the more interest it typically hasfor a viewer. Authors of games could encode values for υ, dependent onthe visible scene, that trigger alternative mappings.

FIG. 25 also highlights an issue with inverted perceived depth close increated with simple scaling. This may be avoided however by forcing allperceived depths closer than the minimum allowed, to be at the minimum.Enforced clipping of Δx′(Z) uses the following logic:

$\begin{matrix}{{\Delta\;{x^{\prime}\left( {\upsilon,Z} \right)}} = {❘\begin{matrix}{{{\upsilon \cdot \Delta}\;{x(Z)}\mspace{14mu}{if}\mspace{14mu}{A \cdot \frac{Z}{\left( {{{2 \cdot \upsilon \cdot \Delta}\;{x(Z)}} + A} \right)}}} > {Z\_ min}} \\{\left\lbrack {\frac{A}{2} \cdot \left( {\frac{Z}{Z\_ min} - 1} \right)} \right\rbrack\mspace{14mu}{otherwise}}\end{matrix}}} & {{Equation}\mspace{14mu} 56}\end{matrix}$

While various embodiments in accordance with the principles disclosedherein have been described above, it should be understood that they havebeen presented by way of example only, and not limitation. Thus, thebreadth and scope of the invention(s) should not be limited by any ofthe above-described exemplary embodiments, but should be defined only inaccordance with any claims and their equivalents issuing from thisdisclosure. Furthermore, the above advantages and features are providedin described embodiments, but shall not limit the application of suchissued claims to processes and structures accomplishing any or all ofthe above advantages.

Additionally, the section headings herein are provided for consistencywith the suggestions under 37 CFR 1.77 or otherwise to provideorganizational cues. These headings shall not limit or characterize theinvention(s) set out in any claims that may issue from this disclosure.Specifically and by way of example, although the headings refer to a“Technical Field,” the claims should not be limited by the languagechosen under this heading to describe the so-called field. Further, adescription of a technology in the “Background” is not to be construedas an admission that certain technology is prior art to any invention(s)in this disclosure. Neither is the “Summary” to be considered as acharacterization of the invention(s) set forth in issued claims.Furthermore, any reference in this disclosure to “invention” in thesingular should not be used to argue that there is only a single pointof novelty in this disclosure. Multiple inventions may be set forthaccording to the limitations of the multiple claims issuing from thisdisclosure, and such claims accordingly define the invention(s), andtheir equivalents, that are protected thereby. In all instances, thescope of such claims shall be considered on their own merits in light ofthis disclosure, but should not be constrained by the headings set forthherein.

What is claimed is:
 1. A method for processing a Computer Generated (CG)stereoscopic scene, the method comprising: receiving a CG model of astereoscopic scene having left image and right image model points; anddetermining repositioning for the left image and right image modelpoints in the CG model from input variables comprising an inputdisparity parameter pair and a scene depth parameter pair; wherein thescene depth parameter pair comprises a depth of the furthest object inthe CG stereoscopic scene and a depth of the closest object in the CGstereoscopic scene, further wherein the depths of the furthest objectand the closest object in the CG stereoscopic scene are determined fromthe CG stereoscopic scene; and wherein the depths of the closest objectand the furthest object in the CG stereoscopic scene remainsubstantially unchanged after repositioning; wherein the input disparityparameter pair comprises D_(max) and D_(min), wherein D_(max) is themaximum disparity limit for parallel eyes, and D_(min) is the maximumdisparity limit for crossed eyes, wherein the scene depth parameter paircomprises Z_(max) and Z_(min), wherein Z_(max) is the furthest object inthe CG stereoscopic scene, and Z_(min) is the closest object in the CGstereoscopic scene; wherein determining repositioning for left image andright image model points comprises using a point reposition depthmapping function comprising:${{Z^{\prime}(Z)} = \frac{\left( {{\alpha \cdot {\ln\left( {Z + \beta} \right)}} + B} \right) \cdot \beta}{1 - {\alpha \cdot {\ln\left( {Z + \beta} \right)}} - B}},{{{wherein}\mspace{14mu}\beta} = \frac{\left( {D_{\max} - D_{\min}} \right) \cdot \left( {Z_{\min} \cdot Z_{\max}} \right)}{\left\lbrack {{E \cdot \left( {Z_{\max} - Z_{\min}} \right)} - {D_{\max} \cdot Z_{\max}} + {D_{\min} \cdot Z_{\min}}} \right\rbrack}},{{{wherein}\mspace{14mu}\alpha} = \frac{\beta \cdot \left( {Z_{\min} - Z_{\max}} \right)}{{\ln\;\left\lbrack \frac{\left( {Z_{\min} + \beta} \right)}{\left. {Z_{\max} + \beta} \right)} \right\rbrack} \cdot \left\lbrack {\left( {Z_{\min} + \beta} \right) \cdot \left( {Z_{\max} + \beta} \right)} \right\rbrack}},{{{wherein}\mspace{14mu} B} = {\frac{Z_{\min}}{Z_{\min} + \beta} - {\alpha \cdot {\ln\left( {Z_{\min} + \beta} \right)}}}},$wherein Z is an initial model point location from a left-eye or aright-eye observation point, wherein ${Z = \begin{pmatrix}x \\y \\z\end{pmatrix}},$ and wherein E is eye separation.
 2. The method of claim1, further comprising determining the input disparity parameter pairfrom the CG model.
 3. The method of claim 1, further comprisingdetermining the input disparity parameter pair from a predeterminedsetting.
 4. The method of claim 1, further comprising determining thescene depth parameter pair from the CG model.
 5. The method of claim 1,further comprising determining the scene depth parameter pair from apredetermined setting.
 6. The method of claim 1, wherein determiningrepositioning for left image and right image model points comprisesusing a point reposition depth mapping function comprising:${{Z^{\prime}(Z)} = \frac{\beta}{\left( {{K \cdot Z^{- \gamma}} - 1} \right)}},{{{wherein}\mspace{14mu}\beta} = \frac{\left( {D_{\max} - D_{\min}} \right) \cdot \left( {Z_{\min} \cdot Z_{\max}} \right)}{\left\lbrack {{E \cdot \left( {Z_{\max} - Z_{\min}} \right)} - {D_{\max} \cdot Z_{\max}} + {D_{\min} \cdot Z_{\min}}} \right\rbrack}},{{{wherein}\mspace{14mu}\gamma} = \frac{\ln\left\lbrack \frac{\left( {E - D_{\max}} \right)}{\left( {E - D_{\min}} \right)} \right\rbrack}{\ln\left( \frac{Z_{\min}}{Z_{\max}} \right)}},{{{wherein}\mspace{14mu} K} = \frac{\left( {Z_{\min} + \beta} \right)}{Z_{\min}^{1 - \gamma}}},$wherein Z is an initial model point location from a left-eye or aright-eye observation point, wherein ${Z = \begin{pmatrix}x \\y \\z\end{pmatrix}},$ and wherein E is eye separation.
 7. The method of claim1, wherein the input disparity parameter pair is based on a minimumseparation and a maximum separation.
 8. The method of claim 1, whereindetermining repositioning comprises referencing a predetermined pointreposition depth mapping from a look up table based on the inputvariables.
 9. The method of claim 1, wherein the input variables furthercomprise a head tilt variable, wherein the stereoscopic CG scene ismodified in accordance with the head tilt variable.
 10. The method ofclaim 1, further comprising rendering the CG modified stereoscopicscene.
 11. An apparatus for processing a computer generated (CG)stereoscopic scene comprising at least one processor implementing:receiving a CG model of a stereoscopic scene having left image and rightimage model points; and repositioning the left image and right imagemodel points in the CG model from input variables comprising an inputdisparity parameter pair and a scene depth parameter pair, wherein thescene depth parameter pair comprises the furthest object in the CGstereoscopic scene and the closest object in the CG stereoscopic scene,and wherein the left and right image model points of the furthest objectand the closest object in the CG stereoscopic scene remainssubstantially unchanged after repositioning further wherein the depthsof the furthest object and the closest object in the CG stereoscopicscene are determined from the CG stereoscopic scene; wherein the inputdisparity parameter pair comprises D_(max) and D_(min), wherein D_(max)is the maximum disparity limit for parallel eyes, and D_(min) is themaximum disparity limit for crossed eyes, wherein the scene depthparameter pair comprises Z_(max) and Z_(min), wherein Z_(max) is thefurthest object in the CG stereoscopic scene, and Z_(min) is the closestobject in the CG stereoscopic scene, wherein the processing subsystemutilizes a point reposition depth mapping function comprising:${{Z^{\prime}(Z)} = \frac{\left( {{\alpha \cdot {\ln\left( {Z + \beta} \right)}} + B} \right) \cdot \beta}{1 - {\alpha \cdot {\ln\left( {Z + \beta} \right)}} - B}},{{{wherein}\mspace{14mu}\beta} = \frac{\left( {D_{\max} - D_{\min}} \right) \cdot \left( {Z_{\min} \cdot Z_{\max}} \right)}{\left\lbrack {{E \cdot \left( {Z_{\max} - Z_{\min}} \right)} - {D_{\max} \cdot Z_{\max}} + {D_{\min} \cdot Z_{\min}}} \right\rbrack}},{{{wherein}\mspace{14mu}\alpha} = \frac{\beta \cdot \left( {Z_{\min} - Z_{\max}} \right)}{{\ln\;\left\lbrack \frac{\left( {Z_{\min} + \beta} \right)}{\left. {Z_{\max} + \beta} \right)} \right\rbrack} \cdot \left\lbrack {\left( {Z_{\min} + \beta} \right) \cdot \left( {Z_{\max} + \beta} \right)} \right\rbrack}},{{{wherein}\mspace{14mu} B} = {\frac{Z_{\min}}{Z_{\min} + \beta} - {\alpha \cdot {\ln\left( {Z_{\min} + \beta} \right)}}}},$wherein Z is an initial model point location from a left-eye or aright-eye observation point, where ${Z = \begin{pmatrix}x \\y \\z\end{pmatrix}},$ and wherein E is eye separation.
 12. The apparatus ofclaim 11 further comprises a buffer.
 13. The apparatus of claim 11,wherein the processor utilizes a point reposition depth mapping functioncomprising:${{Z^{\prime}(Z)} = \frac{\beta}{\left( {{K \cdot Z^{- \gamma}} - 1} \right)}},{{{wherein}\mspace{14mu}\beta} = \frac{\left( {D_{\max} - D_{\min}} \right) \cdot \left( {Z_{\min} \cdot Z_{\max}} \right)}{\left\lbrack {{E \cdot \left( {Z_{\max} - Z_{\min}} \right)} - {D_{\max} \cdot Z_{\max}} + {D_{\min} \cdot Z_{\min}}} \right\rbrack}},{{{wherein}\mspace{14mu}\gamma} = \frac{\ln\left\lbrack \frac{\left( {E - D_{\max}} \right)}{\left( {E - D_{\min}} \right)} \right\rbrack}{\ln\left( \frac{Z_{\min}}{Z_{\max}} \right)}},{{{wherein}\mspace{14mu} K} = \frac{\left( {Z_{\min} + \beta} \right)}{Z_{\min}^{1 - \gamma}}},$wherein Z is an initial model point location from a left-eye or aright-eye observation point, wherein ${Z = \begin{pmatrix}x \\y \\z\end{pmatrix}},$ and wherein E is eye separation.
 14. The apparatus ofclaim 11, wherein the input variables further comprise a head tiltvariable, wherein the stereoscopic CG scene is modified in accordancewith the head tilt variable.